Navigating a UAV having an on-board digital camera to capture desired geographic area

ABSTRACT

Methods, systems, and products for navigating a UAV having an on-board digital camera are provided. Embodiments include identifying a geographic area not captured by the digital camera while the UAV is flying in a current flying pattern, and modifying the current flying pattern to capture an image of the identified geographic area. Identifying a geographic area not captured by the digital camera while the UAV is flying in a current flying pattern may be carried out by determining an area captured by the onboard camera, extrapolating the area captured by the onboard camera along the flying pattern to determine a perimeter of uncaptured geographic area, and determining the area of the uncaptured geographic area in dependence upon the perimeter.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation application of and claims priorityfrom U.S. patent application Ser. No. 11/041,923, filed on Jan. 24,2005.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention is data processing, or, more specifically,methods, systems, and products for navigating a UAV having an on-boarddigital camera.

2. Description of Related Art

Many forms of UAV are available in prior art, both domestically andinternationally. Their payload weight carrying capability, theiraccommodations (volume, environment), their mission profiles (altitude,range, duration), and their command, control and data acquisitioncapabilities vary significantly. Routine civil access to these variousUAV assets is in an embryonic state.

Conventional UAVs are typically manually controlled by an operator whomay view aspects of a UAV's flight using cameras installed on the UAVwith images provided through downlink telemetry. Navigating such UAVsfrom a starting position to one or more waypoints requires an operatorto have specific knowledge of the UAV's flight, including such aspectsas starting location, the UAV's current location, waypoint locations,and so on. Operators of prior art UAVs usually are required generally tomanually control the UAV from a starting position to a waypoint withlittle aid from automation. There is therefore an ongoing need forimprovement in the area of UAV navigations.

SUMMARY OF THE INVENTION

Methods, systems, and products for navigating a UAV having an on-boarddigital camera are provided. Embodiments include identifying ageographic area not captured by the digital camera while the UAV isflying in a current flying pattern, and modifying the current flyingpattern to capture an image of the identified geographic area.Identifying a geographic area not captured by the digital camera whilethe UAV is flying in a current flying pattern may be carried out bydetermining an area captured by the onboard camera, extrapolating thearea captured by the onboard camera along the flying pattern todetermine a perimeter of uncaptured geographic area, and determining thearea of the uncaptured geographic area in dependence upon the perimeter.Identifying a geographic area not captured by the digital camera whilethe UAV is flying in a current flying pattern may also be carried out bydetermining a gap in camera coverage between the camera coverage of theon-board camera of the UAV in the current flying pattern and a cameracoverage of another on-board camera of another UAV in another flyingpattern.

Modifying the current flying pattern to capture an image of theidentified geographic area may include identifying flight controlinstructions for changing the altitude of the UAV. Modifying the currentflying pattern to capture an image of the identified geographic area mayalso include identifying flight control instructions for changing theshape of the current flying pattern of the UAV.

Piloting the UAV in a current flying pattern may include receiving froma GPS receiver a current position of the UAV, calculating a heading independence upon a flying pattern algorithm, and flying on the heading.

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescriptions of exemplary embodiments of the invention as illustrated inthe accompanying drawings wherein like reference numbers generallyrepresent like parts of exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 sets forth a system diagram illustrating relations amongcomponents of an exemplary system for navigating a UAV.

FIG. 2 is a block diagram of an exemplary UAV showing relations amongcomponents of included automated computing machinery.

FIG. 3 is a block diagram of an exemplary remote control device showingrelations among components that includes automated computing machinery.

FIG. 4 sets forth a flow chart illustrating an exemplary method fornavigating a UAV that includes receiving in a remote control device auser's selection of a GUI map pixel that represents a waypoint for UAVnavigation.

FIG. 4A is a data flow diagram illustrating an exemplary method forreceiving downlink telemetry.

FIG. 4B sets forth a data flow diagram illustrating an exemplary methodfor transmitting uplink telemetry.

FIG. 5 sets forth a block diagram that includes a GUI displaying a mapand a corresponding area of the surface of the Earth.

FIG. 6 sets forth a flow chart illustrating an exemplary method ofnavigating a UAV in accordance with a navigation algorithm.

FIG. 7 sets forth a line drawing illustrating a flight path produced byapplication of the method of FIG. 6.

FIG. 8 sets forth a flow chart illustrating an exemplary method ofnavigating a UAV in accordance with a navigation algorithm.

FIG. 9 sets forth a line drawing illustrating a flight path produced byapplication of the method of FIG. 8.

FIG. 10 sets forth a flow chart illustrating an exemplary method ofnavigating a UAV in accordance with a navigation algorithm.

FIG. 11 sets forth a line drawing illustrating a flight path produced byapplication of the method of FIG. 10.

FIG. 12 sets forth a flow chart illustrating an exemplary method fornavigating a UAV that includes receiving in a remote control device auser's selection of a GUI map pixel that represents a waypoint for UAVnavigation.

FIG. 13 sets forth a flow chart illustrating an exemplary method ofpiloting in accordance with a navigation algorithm.

FIG. 14 sets forth a line drawing illustrating a method of calculating aheading with a cross wind to achieve a particular ground course.

FIG. 15 sets forth a line drawing illustrating a flight path produced byapplication of the method of FIG. 13.

FIG. 16 sets forth a flow chart illustrating an exemplary method ofpiloting in accordance with a navigation algorithm.

FIG. 17 sets forth a line drawing illustrating a flight path produced byapplication of the method of FIG. 16.

FIG. 18 sets forth a flow chart illustrating an exemplary method fornavigating a UAV having an on-board digital camera.

FIG. 19 sets forth a flow chart illustrating an exemplary method forflying a pattern.

FIG. 20 sets forth a line drawing illustrating an aerial view of ageographic area uncaptured by a UAV flying a square shaped patternuseful in explaining a method for identifying a geographic area notcaptured by the digital camera while the UAV is flying in a currentflying pattern.

FIG. 21 sets forth a line drawing illustrating an aerial view of ageographic area uncaptured by a UAV flying a square shaped patternuseful in explaining a method for identifying a geographic area notcaptured by the digital camera while the UAV is flying in a currentflying pattern.

FIG. 22 sets forth a line drawing illustrating two UAVs each equippedwith on-board digital cameras and each flying a square shaped flyingpattern.

FIG. 23 sets forth a line drawing illustrating an aerial view ofgeographic areas uncaptured by a two UAVs flying square shaped patternsuseful in explaining a method for identifying a geographic area notcaptured by the digital camera while the UAV is flying in a currentflying pattern in formation with another UAV.

FIG. 24 sets forth line drawing illustrating a changed area of cameracoverage resulting from modifying the flying pattern of the UAV.

FIG. 25 sets forth a line drawing illustrating the changed area ofcamera coverage resulting in changing the flying pattern of the UAV bychanging the shape of the flying pattern of the UAV.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS Introduction

The present invention is described to a large extent in thisspecification in terms of methods for navigating a UAV having anon-board digital camera. Persons skilled in the art, however, willrecognize that any computer system that includes suitable programmingmeans for operating in accordance with the disclosed methods also fallswell within the scope of the present invention. Suitable programmingmeans include any means for directing a computer system to execute thesteps of the method of the invention, including for example, systemscomprised of processing units and arithmetic-logic circuits coupled tocomputer memory, which systems have the capability of storing incomputer memory, which computer memory includes electronic circuitsconfigured to store data and program instructions, and programmed stepsof the method of the invention for execution by a processing unit.

The invention also may be embodied in a computer program product, suchas a diskette or other recording medium, for use with any suitable dataprocessing system. Embodiments of a computer program product may beimplemented by use of any recording medium for machine-readableinformation, including magnetic media, optical media, or other suitablemedia. Persons skilled in the art will immediately recognize that anycomputer system having suitable programming means will be capable ofexecuting the steps of the method of the invention as embodied in aprogram product. Persons skilled in the art will recognize immediatelythat, although most of the exemplary embodiments described in thisspecification are oriented to software installed and executed oncomputer hardware, nevertheless, alternative embodiments implemented asfirmware or as hardware are well within the scope of the presentinvention.

Definitions

“Airspeed” means UAV airspeed, the speed of the UAV through the air.

A “cross track” is a fixed course from a starting point directly to awaypoint. A cross track has a direction, a ‘cross track direction,’ thatis the direction straight from a starting point to a waypoint. That is,a cross track direction is the heading that a UAV would fly directlyfrom a starting point to a waypoint in the absence of wind.

“GUI” means graphical user interface, a display means for a computerscreen.

“Heading” means the compass heading of the UAV.

“Course” means the direction of travel of the UAV over the ground. Inthe absence of wind, or in the presence of a straight tailwind orstraight headwind, the course and the heading are the same direction. Inthe presence of cross wind, the course and the heading are differentdirections.

“Position” refers to a location in the air or over the ground.‘Position’ is typically specified as Earth coordinates, latitude andlongitude. A specification of position may also include altitude.

A “waypoint” is a position chosen as a destination for navigation of aroute. A route has one or more waypoints. That is, a route is composedof waypoints, including at least one final waypoint, and one or moreintermediate waypoints.

“TDMA” stands for Time Division Multiple Access, a technology fordelivering digital wireless service using time-division multiplexing.TDMA works by dividing a radio frequency into time slots and thenallocating slots to multiple calls. In this way, a single frequency cansupport multiple, simultaneous data channels. TDMA is used by GSM.

“GSM” stands for Global System for Mobile Communications, a digitalcellular standard. GSM at this time is the de facto standard forwireless digital communications in Europe and Asia.

“CDPD” stands for Cellular Digital Packet Data, a data transmissiontechnology developed for use on cellular phone frequencies. CDPD usesunused cellular channels to transmit data in packets. CDPD supports datatransfer rates of up to 19.2 Kbps.

“GPRS” stands for General Packet Radio Service, a standard for wirelessdata communications which runs at speeds up to 150 Kbps, compared withcurrent GSM systems which cannot support more than about 9.6 Kbps. GPRS,which supports a wide range of speeds, is an efficient use of limitedbandwidth and is particularly suited for sending and receiving smallbursts of data, such as e-mail and Web browsing, as well as largevolumes of data.

“EDGE” stands for Enhanced Data Rates for GSM Evolution, a standard forwireless data communications supporting data transfer rates of more than300 Kbps. GPRS and EDGE are considered interim steps on the road toUMTS.

“UMTS” stands for Universal Mobile Telecommunication System, a standardfor wireless data communications supporting data transfer rates of up to2 Mpbs. UMTS is also referred to W-CDMA for Wideband Code DivisionMultiple Access.

Exemplary Architecture

Methods, systems, and products for navigating a UAV are explained withreference to the accompanying drawings, beginning with FIG. 1. FIG. 1sets forth a system diagram illustrating relations among components ofan exemplary system for navigating a UAV. The system of FIG. 1 includesUAV (100) which includes a GPS (Global Positioning System) receiver (notshown) that receives a steady stream of GPS data from satellites (190,192). For convenience of explanation, only two GPS satellites are shownin FIG. 1, although the GPS satellite network in fact includes 24 GPSsatellites.

The system of FIG. 1 operates to navigate a UAV by receiving in a remotecontrol device a user's selection of a GUI map pixel that represents awaypoint for UAV navigation. Each such pixel has a location on a GUImap, typically specified as a row and column position. Examples ofremote control devices in FIG. 1 include mobile telephone (110),workstation (104), laptop computer (116), and PDA (Personal DigitalAssistant) (120). Each such remote control device is capable ofsupporting a GUI display of a map of the surface of the Earth in whicheach pixel on the GUI map represents a position on the Earth.

Each remote control device also supports at least one user input devicethrough which a user may enter the user's selection of a pixel. Examplesof user input devices in the system of FIG. 1 include telephone keypad(122), workstation keyboard (114), workstation joystick (112), laptopkeyboard (116) and PDA touch screen (118).

The system of FIG. 1 typically is capable of operating a remote controldevice to map the pixel's location on the GUI to Earth coordinates of awaypoint. The remote control device is often capable of receivingdownlink telemetry including starting position from a GPS receiver onthe UAV through the socket. In fact, the remote control device is oftenreceiving downlink telemetry that includes a steady stream of GPSpositions of the UAV. Receiving a starting position therefore istypically carried out by taking the current position of the UAV when theuser selects the pixel as the starting position. In the example of FIG.1, the remote control device generally receives the starting positionfrom the UAV through wireless network (102). The remote control deviceis often capable of transmitting uplink telemetry including thecoordinates of the waypoint, flight control instructions, or UAVinstructions through a socket on the remote control devices.

Wireless network (102) is implemented using any wireless datatransmission technology as will occur to those of skill in the artincluding, for example, TDMA, GSM, CDPD, GPRS, EDGE, and UMTS. In a oneembodiment, a data communications link layer is implemented using one ofthese technologies, a data communications network layer is implementedwith the Internet Protocol (“IP”), and a data communicationstransmission layer is implemented using the Transmission ControlProtocol (“TCP”). In such systems, telemetry between the UAV and remotecontrol devices, including starting positions, UAV instructions, andflight control instructions, are transmitted using an application-levelprotocol such as, for example, the HyperText Transmission Protocol(“HTTP”), the Wireless Application Protocol (“WAP”), the Handheld DeviceTransmission Protocol (“HDTP”), or any other data communicationsprotocol as will occur to those of skill in the art.

The system of FIG. 1 typically is capable of calculating a heading independence upon the starting position, the coordinates of the waypoint,and a navigation algorithm, identifying flight control instructions forflying the UAV on the heading, and transmitting the flight controlinstructions from the remote control device to the UAV.

The system of FIG. 1 is also capable of navigating the UAV (100) independence upon the digital resolution of an on-board digital camera(552). The system of FIG. 1 is capable of identifying a geographic areanot captured by the digital camera while the UAV is flying in a currentflying pattern and modifying the current flying pattern to capture animage of the identified geographic area. A flying pattern is aconsistent pattern of flight of a UAV. Flying patterns include patternsfor orbiting a waypoint, flying a square or other shape around awaypoint, or other flying patterns that will occur to those of skill inthe art. A flying pattern typically is implemented by a consistentseries of flight control instructions that pilot the UAV such that theresulting flight path creates a pattern. Flying patterns are oftenimplemented with algorithms that result in the UAV flying in aparticular shaped pattern over the ground at a particular altitude.

Commercial off-the-shelf high resolution digital cameras for use innavigating UAVs are currently available. One example of such a highresolution digital camera capable of being mounted on a UAV is the ProBack digital camera available from Kodak.

UAVs according to embodiments of the present invention typicallyinclude, not only an aircraft, but also automated computing machinerycapable of receiving GPS data, operating telemetry between the UAV andone or more remote control devices, and navigating a UAV amongwaypoints. FIG. 2 is a block diagram of an exemplary UAV showingrelations among components of included automated computing machinery. InFIG. 2, UAV (100) includes a processor (164), also typically referred toas a central processing unit or ‘CPU.’ The processor may be amicroprocessor, a programmable control unit, or any other form ofprocessor useful according to the form factor of a particular UAV aswill occur to those of skill in the art. Other components of UAV (100)are coupled for data transfer to processor (164) through system bus(160).

UAV (100) includes random access memory or ‘RAM’ (166). Stored in RAM(166) is an application program (158) that implements inventive methodsaccording to embodiments of the present invention. In some embodiments,the application programming runs on an OSGi services framework (156).OSGi Stands for ‘Open Services Gateway Initiative.’ The OSGispecification is a Java-based application layer framework that providesvendor neutral application layer APIs and functions. An OSGi serviceframework (156) is written in Java and therefore typically runs on aJava Virtual Machine (JVM) (154) which in turn runs on an operatingsystem (150). Examples of operating systems useful in UAVs according tothe present invention include Unix, AIX™, and Microsoft Windows™.

In OSGi, the framework is a hosting platform for running ‘services’.Services are the main building blocks for creating applicationsaccording to the OSGi. A service is a group of Java classes andinterfaces that implement a certain feature. The OSGi specificationprovides a number of standard services. For example, OSGi provides astandard HTTP service that can respond to requests from HTTP clients,such as, for example, remote control devices according to embodiments ofthe present invention. That is, such remote control devices are enabledto communicate with a UAV having an HTTP service by use of datacommunications messages in the HTTP protocol.

Services in OSGi are packaged in ‘bundles’ with other files, images, andresources that the services need for execution. A bundle is a Javaarchive or ‘JAR’ file including one or more service implementations, anactivator class, and a manifest file. An activator class is a Java classthat the service framework uses to start and stop a bundle. A manifestfile is a standard text file that describes the contents of the bundle.

The services framework in OSGi also includes a service registry. Theservice registry includes a service registration including the service'sname and an instance of a class that implements the service for eachbundle installed on the framework and registered with the serviceregistry. A bundle may request services that are not included in thebundle, but are registered on the framework service registry. To find aservice, a bundle performs a query on the framework's service registry.

The application program (158) of FIG. 2 is capable generally ofnavigating a UAV in dependence upon the digital resolution of anon-board digital camera. The system of FIG. 1 is capable of identifyinga geographic area not captured by the digital camera while the UAV isflying in a current flying pattern and modifying the current flyingpattern to capture an image of the identified geographic area.

In the UAV (100) of FIG. 2, software programs and other usefulinformation may be stored in RAM or in non-volatile memory (168).Non-volatile memory (168) may be implemented as a magnetic disk drivesuch as a micro-drive, an optical disk drive, static read only memory(‘ROM’), electrically erasable programmable read-only memory space(‘EEPROM’ or ‘flash’ memory), or otherwise as will occur to those ofskill in the art.

UAV (100) includes communications adapter (170) implementing datacommunications connections (184) to other computers (162), which may bewireless networks, satellites, remote control devices, servers, orothers as will occur to those of skill in the art. Communicationsadapter (170) advantageously facilitates receiving flight controlinstructions from a remote control device. Communications adaptersimplement the hardware level of data communications connections throughwhich UAVs transmit wireless data communications. Examples ofcommunications adapters include wireless modems for dial-up connectionsthrough wireless telephone networks.

UAV (100) includes servos (178). Servos (178) are proportional controlservos that convert digital control signals from system bus (160) intoactual proportional displacement of flight control surfaces, ailerons,elevators, and the rudder. The displacement of flight control surfacesis ‘proportional’ to values of digital control signals, as opposed tothe ‘all or nothing’ motion produced by some servos. In this way,ailerons, for example, may be set to thirty degrees, sixty degrees, orany other supported angle rather than always being only neutral or fullyrotated. Several proportional control servos useful in various UAVsaccording to embodiments of the present invention are available fromFutaba®.

UAV (100) includes a servo control adapter (172). A servo controladapter (172) is multi-function input/output servo motion controllercapable of controlling several servos. An example of such a servocontrol adapter is the “IOSERVO” model from National Control Devices ofOsceola, Mo. The IOSERVO is described on National Control Deviceswebsite at www.controlanything.com.

UAV (100) includes a flight stabilizer system (174). A flight stabilizersystem is a control module that operates servos (178) to automaticallyreturn a UAV to straight and level flight, thereby simplifying the workthat must be done by navigation algorithms. An example of a flightstabilizer system useful in various embodiments of UAVs according to thepresent invention is model CO-Pilot™ from FMA, Inc., of Frederick, Md.The Co-Pilot flight stabilizer system identifies a horizon with heatsensors, identifies changes in aircraft attitude relative to thehorizon, and sends corrective signals to the servos (178) to keep theUAV flying straight and level.

UAV (100) includes an AVCS gyro (176). An AVCS gryo is an angular vectorcontrol system gyroscope that provides control signal to the servos tocounter undesired changes in attitude such as those caused by suddengusts of wind. An example of an AVCS gyro useful in various UAVsaccording to the present invention is model GYA350 from Futaba®.

The UAV (100) of FIG. 2 includes an on-board digital camera (552).Commercial off-the-shelf high resolution digital cameras for use innavigating UAVs are currently available. One example of such a highresolution digital camera capable of being mounted on a UAV is the ProBack digital camera available from Kodak.

Remote control devices according to embodiments of the present inventiontypically include automated computing machinery capable of receivinguser selections of pixel on GUI maps, mapping the pixel to a waypointlocation, receiving downlink telemetry including for example a startingposition from a GPS receiver on the UAV, calculating a heading independence upon the starting position, the coordinates of the waypoint,and a navigation algorithm, identifying flight control instructions forflying the UAV on the heading, and transmitting the flight controlinstructions as uplink telemetry from the remote control device to theUAV. FIG. 3 is a block diagram of an exemplary remote control deviceshowing relations among components of included automated computingmachinery. In FIG. 3, remote control device (161) includes a processor(164), also typically referred to as a central processing unit or ‘CPU.’The processor may be a microprocessor, a programmable control unit, orany other form of processor useful according to the form factor of aparticular remote control device as will occur to those of skill in theart. Other components of remote control device (161) are coupled fordata transfer to processor (164) through system bus (160).

Remote control device (161) includes random access memory or ‘RAM’(166). Stored in RAM (166) an application program (152) that implementsinventive methods of the present invention. In some embodiments, theapplication program (152) is OSGi compliant and therefore runs on anOSGi services framework installed (not shown) on a JVM (not shown). Inaddition, software programs and further information for use inimplementing methods of navigating a UAV according to embodiments of thepresent invention may be stored in RAM or in non-volatile memory (168).

The application program (152) of FIG. 3 is capable generally ofnavigating a UAV in dependence upon the digital resolution of anon-board digital camera. The system of FIG. 1 is capable of identifyinga geographic area not captured by the digital camera while the UAV isflying in a current flying pattern and modifying the current flyingpattern to capture an image of the identified geographic area bytransmitting flight control instructions to the UAV to modify thecurrent flying pattern.

Non-volatile memory (168) may be implemented as a magnetic disk drivesuch as a micro-drive, an optical disk drive, static read only memory(‘ROM’), electrically erasable programmable read-only memory space(‘EEPROM’ or ‘flash’ memory), or otherwise as will occur to those ofskill in the art.

Remote control device (161) includes communications adapter (170)implementing data communications connections (184) to other computers(162), including particularly computers on UAVs. Communications adaptersimplement the hardware level of data communications connections throughwhich remote control devices communicate with UAVs directly or throughnetworks. Examples of communications adapters include modems for wireddial-up connections, Ethernet (IEEE 802.3) adapters for wired LANconnections, 802.11b adapters for wireless LAN connections, andBluetooth adapters for wireless microLAN connections.

The example remote control device (161) of FIG. 3 includes one or moreinput/output interface adapters (180). Input/output interface adaptersin computers implement user-oriented input/output through, for example,software drivers and computer hardware for controlling output to displaydevices (185) such as computer display screens, as well as user inputfrom user input devices (182) such as keypads, joysticks, keyboards, andtouch screens.

Navigating a UAV

FIG. 4 sets forth a flow chart illustrating an exemplary method fornavigating a UAV that includes receiving (402) in a remote controldevice a user's selection of a GUI map pixel (412) that represents awaypoint for UAV navigation. The pixel has a location on the GUI. Such aGUI map display has many pixels, each of which represents at least oneposition on the surface of the Earth. A user selection of a pixel isnormal GUI operations to take a pixel location, row and column, from aGUI input/output adapter driven by a user input device such as ajoystick or a mouse. The remote control device can be a traditional‘ground control station,’ an airborne PDA or laptop, a workstation inEarth orbit, or any other control device capable of accepting userselections of pixels from a GUI map.

The method of FIG. 4 includes mapping (404) the pixel's location on theGUI to Earth coordinates (414) of the waypoint. As discussed in moredetail above with reference to FIG. 5, mapping (404) the pixel'slocation on the GUI to Earth coordinates of the waypoint (414) typicallyincludes mapping pixel boundaries of the GUI map to corresponding Earthcoordinates and identifying a range of latitude and a range of longituderepresented by each pixel. Mapping (404) the pixel's location on the GUIto Earth coordinates of the waypoint (414) also typically includeslocating a region on the surface of the Earth in dependence upon theboundaries, the ranges, and the location of the pixel on the GUI map.

The method of FIG. 4 also includes receiving (408) downlink telemetry,including a starting position from a GPS receiver on the UAV, from theUAV through a socket on the remote control device. In fact, the remotecontrol device is receiving downlink telemetry that includes a steadystream of GPS positions of the UAV. Receiving a starting positiontherefore is typically carried out by taking the current position of theUAV when the user selects the pixel as the starting position.

A socket is one end-point of a two-way communication link between twoapplication programs running on a network. In Java, socket classes areused to represent a connection between a client program and a serverprogram. The java.net package provides two Java classes—Socket andServerSocket—that implement the client side of the connection and theserver side of the connection, respectively. In some embodiments of thepresent invention, a Java web server, is included in an OSGi frameworkon a remote control device. Often then, a socket on the remote controldevice would be considered a server-side socket, and a socket on the UAVwould be considered a client socket. In other embodiments of the presentinvention, a Java web server, is included in an OSGi framework on theUAV. In such embodiments, a socket on the UAV would be considered aserver-side socket, and a socket on a remote control device would beconsidered a client socket.

Use of a socket requires creating a socket and creating data streams forwriting to and reading from the socket. One way of creating a socket andtwo data streams for use with the socket is shown in the followingexemplary pseudocode segment:

uavSocket = new Socket( “computerAddress”, 7); outStream = newPrintWriter(uavSocket.getOutputStream( ), true); inStream = newBufferedReader(new InputStreamReader(uavSocket.getInputStream( )));

The first statement in this segment creates a new socket object andnames it “uavSocket.” The socket constructor used here requires a fullyqualified IP address of the machine the socket is to connect to, in thiscase the Java server on a remote control device or a UAV, and the portnumber to connect to. In this example, “computerAddress” is taken as adomain name that resolves to a fully qualified dotted decimal IPaddress. Alternatively, a dotted decimal IP address may be employeddirectly, as, for example, “195.123.001.001.” The second argument in thecall to the socket constructor is the port number. Port number 7 is theport on which the server listens in this example, whether the server ison a remote control device or on a UAV.

The second statement in this segment gets the socket's output stream andopens a Java PrintWriter object on it. Similarly, the third statementgets the socket's input stream and opens a Java BufferedReader object onit. To send data through the socket, an application writes to thePrintWriter, as, for example:

-   -   outStream.println(someWaypoint, macro, or Flight Control        Instruction);

To receive data through the socket, an application reads from theBufferedReader, as show here for example:

-   -   a Waypoint, GPS data, macro, or flight control        instruction=inStream.readLine( );

The method of FIG. 4 also includes calculating (410) a heading independence upon the starting position, the coordinates of the waypoint,and a navigation algorithm. Methods of calculating a heading arediscussed in detail below in this specification. The method of FIG. 4includes identifying (418) flight control instructions for flying theUAV on the heading. Flight control instructions are specific commandsthat affect the flight control surfaces of the UAV. That is,instructions to move the flight control surfaces to affect the UAV'sflight causing the UAV to turn, climb, descend, and so on. As an aid tofurther explanation, an exemplary method of identifying flight controlinstructions for flying on a calculated heading is provided:

-   -   receive new calculated heading from navigation algorithms    -   read current heading from downlink telemetry    -   if current heading is left of the calculated heading, identify        flight control instruction: AILERONS LEFT 30 DEGREES    -   if current heading is right of the calculated heading, identify        flight control instruction: AILERONS RIGHT 30 DEGREES    -   monitor current heading during turn    -   when current heading matches calculated heading, identify flight        control instruction: FLY STRAIGHT AND LEVEL

The method of FIG. 4 includes transmitting (420) uplink telemetry,including the flight instructions, through the socket to the UAV.Transmitting (420) the flight control instructions from the remotecontrol device to the UAV may be carried out by use of any datacommunications protocol, including, for example, transmitting the flightcontrol instructions as form data, URI encoded data, in an HTTP message,a WAP message, an HDML message, or any other data communicationsprotocol message as will occur to those of skill in the art.

FIG. 4A is a data flow diagram illustrating an exemplary method forreceiving downlink telemetry. The method of FIG. 4A includes listening(450) on the socket (456) for downlink data (458). Listening on a socketfor downlink data may be implemented by opening a socket, creating aninput stream for the socket, and reading data from the input stream, asillustrated, for example, in the following segment of pseudocode:

uavSocket = new Socket( “computerAddress”, 7); inStream = newBufferedReader(new InputStreamReader(uavSocket.getInputStream( )));String downLinkData = inStream.readLine( );

This segment opens a socket object named “uavSocket” with an inputstream named “inStream.” Listening for downlink data on the socket isaccomplished with a blocking call to inStream.readLine( ) which returnsa String object name “downLinkData.”

The method of FIG. 4A includes storing (452) downlink data (458) incomputer memory (166) and exposing (454) the stored downlink data (458)through an API (462) to a navigation application (460). Downlink datatypically is exposed through an ‘API’ (Application ProgrammingInterface) by providing in a Java interface class public accessorfunctions for reading from member data elements in which the downlinkdata is stored. A navigation application wishing to access downlink datathen may access the data by calling a public accessor methods, as, forexample: String someDownLinkData=APIimpl.getDownLinkData( ).

In the method of FIG. 4A, the downlink telemetry (470) further comprisesflight control instructions. It is counterintuitive that downlinktelemetry contains flight control instruction when the expected datacommunications direction for flight control instructions ordinarily isin uplink from a remote control device to a UAV. It is useful to note,however, that flight control instructions can be uplinked from amultiplicity of remote control devices, not just one. A flight linetechnician with a handheld PDA can issue flight control instructions toa UAV that is also linked for flight control to a computer in a groundstation. It is sometimes advantageous, therefore, for downlink telemetryto include flight control instructions so that one remote control devicecan be advised of the fact that some other remote control device issuedflight control instructions to the same UAV.

FIG. 4B sets forth a data flow diagram illustrating an exemplary methodfor transmitting uplink telemetry. The method of FIG. 4B includesmonitoring (466) computer memory (166) for uplink data (464) from anavigation application (460). When uplink data (464) is presented, themethod of FIG. 4B includes sending (468) the uplink data through thesocket (456) to the UAV (100). Sending uplink data through a socket maybe implemented by opening a socket, creating an output stream for asocket, and writing the uplink data to the output stream, asillustrated, for example, in the following segment of pseudocode:

uavSocket = new Socket( “computerAddress”, 7); outStream = newPrintWriter(uavSocket.getOutputStream( ), true);outStream.println(String someUplinkData);

This segment opens a socket object named “uavSocket” with an outputstream named “outStream.” Sending uplink data through the socket isaccomplished with a call to outStream.println( ) which takes as a callparameter a String object named “someUplinkData.”

Macros

Although the flow chart of FIG. 4 illustrates navigating a UAV to asingle waypoint, as a practical matter, embodiments of the presentinvention typically support navigating a UAV along a route having manywaypoints, including a final waypoint and one or more intermediatewaypoints. That is, methods of the kind illustrated in FIG. 4 may alsoinclude receiving user selections of a multiplicity of GUI map pixelsrepresenting waypoints, where each pixel has a location on the GUI andmapping each pixel location to Earth coordinates of a waypoint.

Such methods for navigating a UAV can also include assigning one or moreUAV instructions to each waypoint and storing the coordinates of thewaypoints and the UAV instructions in computer memory on the remotecontrol device. A UAV instruction typically includes one or moreinstructions for a UAV to perform a task in connection with a waypoint.Exemplary tasks include turning on or off a camera installed on the UAV,turning on or off a light installed on the UAV, orbiting a waypoint, orany other task that will occur to those of skill in the art.

UAV instructions to perform tasks in connection with a waypoint may beencoded in, for example, XML (the eXtensible Markup Language) as shownin the following exemplary XML segment:

<UAV-Instructions> <macro> <waypoint> 33° 44′ 10″ N 30° 15′ 50″ W</waypoint> <instruction> orbit </instruction> <instruction>videoCameraON </instruction> <instruction> wait30minutes </instruction><instruction> videoCameraOFF </instruction> <instruction> nextWaypoint</instruction> </macro> <macro> </macro> <macro> </macro> <macro></macro> <UAV-instructions>

This XML example has a root element named ‘UAV-instructions.’ Theexample contains several subelements named ‘macro.’ One ‘macro’subelement contains a waypoint location representing an instruction tofly to 33° 44′10″ N 30° 15′50″ W. That macro subelement also containsseveral instructions for tasks to be performed when the UAV arrives atthe waypoint coordinates, including orbiting around the waypointcoordinates, turning on an on-board video camera, continuing to orbitfor thirty minutes with the camera on, turning off the video camera, andcontinuing to a next waypoint. Only one macro set of UAV instructions isshown in this example, but that is not a limitation of the invention. Infact, such sets of UAV instructions may be of any useful size as willoccur to those of skill in the art.

Exemplary methods of navigating a UAV also include flying the UAV toeach waypoint in accordance with one or more navigation algorithms andoperating the UAV at each waypoint in accordance with the UAVinstructions for each waypoint. Operating the UAV at the waypoint inaccordance with the UAV instructions for each waypoint typicallyincludes identifying flight control instructions in dependence upon theUAV instructions for each waypoint and transmitting the flight controlinstructions as uplink telemetry through a socket. Flight controlinstructions identified in dependence upon the UAV instructions for eachwaypoint typically include specific flight controls to move the flightcontrol surfaces of the UAV causing the UAV to fly in accordance withthe UAV instructions. For example, in the case of a simple orbit, aflight control instruction to move the ailerons and hold them at acertain position causing the UAV to bank at an angle can effect an orbitaround a waypoint.

Operating the UAV at the waypoint in accordance with the UAVinstructions for each way point typically includes transmitting theflight control instructions as uplink data from the remote controldevice to the UAV. Transmitting the flight control instructions asuplink data from the remote control device to the UAV may be carried outby use of any data communications protocol, including, for example,transmitting the flight control instructions as form data, URI encodeddata, in an HTTP message, a WAP message, an HDML message, or any otherdata communications protocol message as will occur to those of skill inthe art.

Pixel Mapping

For further explanation of the process of mapping pixels' locations toEarth coordinates, FIG. 5 sets forth a block diagram that includes a GUI(502) displaying a map (not shown) and a corresponding area of thesurface of the Earth (504). The GUI map has pixel boundaries identifiedas Row₁, Col₁; Row₁, Col₁₀₀; Row₁₀₀, Col₁₀₀; and Row₁₀₀, Col₁. In thisexample, the GUI map is assumed to include 100 rows of pixels and 100columns of pixels. This example of 100 rows and columns is presented forconvenience of explanation; it is not a limitation of the invention. GUImaps according to embodiments of the present invention may include anynumber of pixels as will occur to those of skill in the art.

The illustrated area of the surface of the Earth has correspondingboundary points identified as Lat₁, Lon₁; Lat₁, Lon₂; Lat₂, Lon₂; andLat₂, Lon₁. This example assumes that the distance along one side ofsurface area (504) is 100 nautical miles, so that the distance expressedin terms of latitude or longitude between boundary points of surfacearea (504) is 100 minutes or 1° 40′.

In typical embodiments, mapping a pixel's location on the GUI to Earthcoordinates of a waypoint includes mapping pixel boundaries of the GUImap to Earth coordinates. In this example, the GUI map boundary at Row₁,Col₁ maps to the surface boundary point at Lat₁, Lon₁; the GUI mapboundary at Row₁, Col₂ maps to the surface boundary point at Lat₁, Lon₂;the GUI map boundary at Row₂, Col₂ maps to the surface boundary point atLat₂, Lon₂; the GUI map boundary at Row₂, Col₁ maps to the surfaceboundary point at Lat₂, Lon₁.

Mapping a pixel's location on the GUI to Earth coordinates of a waypointtypically also includes identifying a range of latitude and a range oflongitude represented by each pixel. The range of latitude representedby each pixel may be described as (Lat₂−Lat₁)/N_(rows), where(Lat₂−Lat₁) is the length in degrees of the vertical side of thecorresponding surface (504), and N_(rows) is the number of rows ofpixels. In this example, (Lat₂−Lat₁) is 1° 40′ or 100 nautical miles,and N_(rows) is 100 rows of pixels. The range of latitude represented byeach pixel in this example therefore is one minute of arc or onenautical mile.

Similarly, the range of longitude represented by each pixel may bedescribed as (Lon₂−Lon₁)/N_(cols), where (Lon₂−Lon₁) is the length indegrees of the horizontal side of the corresponding surface (504), andN_(cols) is the number of columns of pixels. In this example,(Lon₂−Lon₁) is 1° 40′ or 100 nautical miles, and N_(cols) is 100 columnsof pixels. The range of longitude represented by each pixel in thisexample therefore is one minute of arc or one nautical mile.

Mapping a pixel's location on the GUI to Earth coordinates of a waypointtypically also includes locating a region on the surface of the Earth independence upon the boundaries, the ranges, and the location of thepixel on the GUI map. The region is the portion of the surfacecorresponding to the pixel itself. That region is located generally bymultiplying in both dimensions, latitude and longitude, the range oflatitude and longitude by column or row numbers of the pixel location onthe GUI map. That is, a latitude for the surface region of interest isgiven by Expression 1.Lat₁ +P _(row)((Lat₂−Lat₁)/N _(rows))  (Exp. 1)In Expression 1:

-   -   Lat₁ is the latitude of an origin point for the surface area        (504) corresponding generally to the GUI map,    -   P_(row) is the row number of the pixel location on the GUI map,        and    -   ((Lat₂−Lat₁)/N_(rows)) is the range of latitude represented by        the pixel.

Similarly, a longitude for the surface region of interest is given byExpression 2.Lon₁+P_(col)((Lon₂−Lon₁)/N_(cols))  (Exp. 2)In Expression 2:

-   -   Lon₁ is the longitude of an origin point for the surface area        (504) corresponding generally to the GUI map,    -   P_(col) is the column number of the pixel location on the GUI        map, and    -   ((Lon₂−Lon₁)/N_(cols)) is the range of longitude represented by        the pixel.

Referring to FIG. 5 for further explanation, Expressions 1 and 2 takentogether identify a region (508) of surface area (504) that correspondsto the location of pixel (412) mapping the pixel location to the bottomleft corner (506) of the region (508). Advantageously, however, manyembodiments of the present invention further map the pixel to the centerof the region by adding one half of the length of the region's sides tothe location of the bottom left corner (506).

More particularly, locating a region on the surface of the Earth independence upon the boundaries, the ranges, and the location of thepixel on the GUI map, as illustrated by Expression 3, may includemultiplying the range of longitude represented by each pixel by a columnnumber of the selected pixel, yielding a first multiplicand; andmultiplying the range of longitude represented by each pixel by 0.5,yielding a second multiplicand; adding the first and secondmultiplicands to an origin longitude of the GUI map.Lon₁ +P _(col)((Lon₂−Lon₁)/N _(cols))+0.5((Lon₂−Lon₁)/N _(cols))  (Exp.3)

In Expression 3, the range of longitude represented by each pixel isgiven by ((Lon₂−Lon₁)/N_(cols)), and the first multiplicand is P_(col)((Lon₂−Lon₁)/N_(cols)). The second multiplicand is given by0.5((Lon₂−Lon₁)/N_(cols)).

Similarly, locating a region on the surface of the Earth in dependenceupon the boundaries, the ranges, and the location of the pixel on theGUI map, as illustrated by Expression 4, typically also includesmultiplying the range of latitude represented by each pixel by a rownumber of the selected pixel, yielding a third multiplicand; multiplyingthe range of latitude represented by each pixel by 0.5, yielding afourth multiplicand; and adding the third and fourth multiplicands to anorigin latitude of the GUI map.Lat₁ +P _(row)((Lat₂−Lat₁)/N _(rows))+0.5((Lat₂−Lat₁)/N _(rows))  (Exp.4)

In Expression 4, the range of latitude represented by each pixel isgiven by ((Lat₂−Lat₁)/N_(rows)), and the third multiplicand isP_(row)((Lat₂−Lat₁)/N_(rows)). The fourth multiplicand is given by0.5((Lat₂−Lat₁)/N_(rows)). Expressions 3 and 4 taken together map thelocation of pixel (412) to the center (510) of the located region (508).

Navigation on a Heading to a Waypoint

An exemplary method of navigating in accordance with a navigationalgorithm is explained with reference to FIGS. 6 and 7. FIG. 6 setsforth a flow chart illustrating an exemplary method of navigating a UAVin accordance with a navigation algorithm, and FIG. 7 sets forth a linedrawing illustrating a flight path produced by application of the methodof FIG. 6.

The method of FIG. 6 includes periodically repeating (610) the steps of,receiving (602) in the remote control device from the GPS receiver acurrent position of the UAV, and calculating (604) a new heading fromthe current position to the waypoint. The method of FIG. 6 also includesidentifying (606) flight control instructions for flying the UAV on thenew heading, and transmitting (608), from the remote control device tothe UAV, the flight control instructions for flying the UAV on the newheading. In this method, if Lon₁, Lat₁ is taken as the current position,and Lon₂, Lat₂ is taken as the waypoint position, then the new headingmay be calculated generally as the inverse tangent of((Lat₂−Lat₁)/(Lon₂−Lon₁)).

FIG. 7 shows the effect of the application of the method of FIG. 6. Inthe example of FIG. 7, a UAV is flying in a cross wind having cross windvector (708). Curved flight path (716) results from periodiccalculations according to the method of FIG. 6 of a new heading straightfrom a current location to the waypoint. FIG. 7 shows periodicrepetitions of the method of FIG. 6 at plot points (710, 712, 714). Forclarity of explanation, only three periodic repetitions are shown,although that is not a limitation of the invention. In fact, any numberof periodic repetitions may be used as will occur to those of skill inthe art.

Navigation with Headings Set to a Cross Track Direction

A further exemplary method of navigating in accordance with a navigationalgorithm is explained with reference to FIGS. 8 and 9. FIG. 8 setsforth a flow chart illustrating an exemplary method of navigating a UAVin accordance with a navigation algorithm, and FIG. 9 sets forth a linedrawing illustrating a flight path produced by application of the methodof FIG. 8. The method of FIG. 8 includes identifying (802) a cross trackbetween the starting point and the waypoint. A cross track is a fixedcourse from a starting point directly to a waypoint. If Lon₁, Lat₁ istaken as the position of a starting point, and Lon₂, Lat₂ is taken asthe waypoint position, then a cross track is identified by Lon₁, Lat₁and Lon₂, Lat₂. A cross track has a direction, a ‘cross trackdirection,’ that is the direction straight from a starting point to awaypoint, and it is often useful to characterize a cross track by itscross track direction. The cross track direction for a cross trackidentified by starting point Lon₁, Lat₁ and waypoint position Lon₂, Lat₂may be calculated generally as the inverse tangent of((Lat₂−Lat₁)/(Lon₂−Lon₁)).

The method of FIG. 8 includes periodically repeating (810) the steps ofreceiving (804) in the remote control device from the GPS receiver acurrent position of the UAV, and calculating (806) a shortest distancebetween the current position and the cross track. If the shortestdistance between the current position and the cross track is greaterthan a threshold distance (808), the method of FIG. 8 includestransmitting (812) flight control instructions that pilot the UAV towardthe cross track, and, when the UAV arrives at the cross track,transmitting (814) flight control instructions that pilot the UAV in across track direction toward the waypoint.

FIG. 9 illustrates calculating a shortest distance between the currentposition and a cross track. In the example of FIG. 9, calculating ashortest distance between the current position and a cross trackincludes calculating the distance from a current position (912) to thewaypoint (704). In the example of FIG. 9, the distance from the currentposition (912) to the waypoint (704) is represented as the length ofline (914). For current position Lon₁, Lat₁, and waypoint position Lon₂,Lat₂, the distance from a current position (912) to the waypoint (704)is given by the square root of (Lat₂−Lat₁)²+(Lon₂−Lon₁)².

In this example, calculating a shortest distance between the currentposition and a cross track also includes calculating the angle (910)between a direction from the current position (912) to the waypoint(704) and a cross track direction. In the example of FIG. 9, thedirection from the current position (912) to the waypoint (704) isrepresented as the direction of line (914). In the example of FIG. 9,the cross track direction is the direction of cross track (706). Theangle between a direction from the current position to the waypoint anda cross track direction is the difference between those directions.

In the current example, calculating a shortest distance between thecurrent position and a cross track also includes calculating the tangentof the angle between a direction from the current position to thewaypoint and a cross track direction and multiplying the tangent of theangle by the distance from the current position to the waypoint.

FIG. 9 also shows the effect of the application of the method of FIG. 8.In the example of FIG. 9, a UAV is flying in a cross wind having crosswind vector (708). The flight path (904) results from periodiccalculations according to the method of FIG. 8 of a shortest distancebetween a current position and the cross track (706), flying the UAVback to the cross track and then flying in the direction of the crosstrack whenever the distance from the cross track exceeds a predeterminedthreshold (916) distance.

Headings Set to Cross Track Direction with Angular Thresholds

A further exemplary method of navigating in accordance with a navigationalgorithm is explained with reference to FIGS. 10 and 11. FIG. 10 setsforth a flow chart illustrating an exemplary method of navigating a UAVin accordance with a navigation algorithm, and FIG. 11 sets forth a linedrawing illustrating a flight path produced by application of the methodof FIG. 10.

In the method of FIG. 10, piloting in accordance with a navigationalgorithm includes identifying (1002) a cross track having a cross trackdirection between the starting point and the waypoint. As describedabove, a cross track is identified by a position of a starting point anda waypoint position. For a starting point position of Lon₁, Lat₁ and awaypoint position of Lon₂, Lat₂, a cross track is identified by Lon₁,Lat₁ and Lon₂, Lat₂. In addition, it is often also useful tocharacterize a cross track by its cross track direction. The cross trackdirection for a cross track identified by starting point Lon₁, Lat₁ andwaypoint position Lon₂, Lat₂ may be calculated generally as the inversetangent of ((Lat₂−Lat₁)/(Lon₂−Lon₁)).

In the method of FIG. 10, navigating a UAV in accordance with anavigation algorithm includes periodically repeating (1010) the steps ofreceiving (1004) in the remote control device from the GPS receiver acurrent position and a current heading of the UAV, and calculating(1006) an angle between the direction from the current position to thewaypoint and a cross track direction. If the angle is greater than athreshold angle (1008), the method of FIG. 10 includes transmitting(1012) flight control instructions that pilot the UAV toward the crosstrack, and, upon arriving at the cross track, transmitting (1014) flightcontrol instructions that pilot the UAV in the cross track directiontoward the waypoint.

Transmitting (1012) flight control instructions that pilot the UAVtoward the cross track is carried out by transmitting flight controlinstructions to turn to a heading no more than ninety degrees from thecross track direction, turning to the left if the current position isright of the cross track and to the right if the current position isleft of the cross track. Transmitting (1014) flight control instructionsthat pilot the UAV in the cross track direction toward the waypointtransmitting flight control instructions to turn the UAV to the crosstrack direction and then flying straight and level on the cross trackdirection.

FIG. 11 shows the effect of the application of the method of FIG. 10. Inthe example of FIG. 11, a UAV is flying in a cross wind having crosswind vector (708). The flight path (1104) results from periodicallytransmitting flight control instructions to fly the UAV, according tothe method of FIG. 10, back to the cross track and then in the directionof the cross track whenever an angle between the direction from thecurrent position to the waypoint and a cross track direction exceeds apredetermined threshold angle.

In many embodiments of the method of FIG. 10, the threshold angle is avariable whose value varies in dependence upon a distance between theUAV and the waypoint. In typical embodiments that vary the thresholdangle, the threshold angle is increased as the UAV flies closer to thewaypoint. It is useful to increase the threshold angle as the UAV fliescloser to the waypoint to reduce the risk of excessive ‘hunting.’ Thatis, because the heading is the cross track direction, straight to the WPrather than cross wind, if the angle remains the same, the distance thatthe UAV needs to be blown off course to trigger transmitting flightcontrol signals instructing the UAV to return to the cross track getssmaller and smaller until the UAV is flying to the cross track, turningto the cross track direction, getting blown immediately across thethreshold, flying back the cross track, turning to the cross trackdirection, getting blown immediately across the threshold, and so on,and so on, in rapid repetition. Increasing the threshold angle as theUAV flies closer to the waypoint increases the lateral distanceavailable for wind error before triggering the transmission of flightinstructions to return to the cross track, thereby reducing this risk ofexcessive hunting.

FIG. 12 sets forth a flow chart illustrating an exemplary method fornavigating a UAV that includes receiving (402) in a remote controldevice a user's selection of a GUI map pixel (412) that represents awaypoint for UAV navigation. The pixel has a location on the GUI. Such aGUI map display has many pixels, each of which represents at least oneposition on the surface of the Earth. A user selection of a pixel isnormal GUI operations to take a pixel location, row and column, from aGUI input/output adapter driven by a user input device such as ajoystick or a mouse. The remote control device can be a traditional‘ground control station,’ an airborne PDA or laptop, a workstation inEarth orbit, or any other control device capable of accepting userselections of pixels from a GUI map.

The method of FIG. 12 includes mapping (404) the pixel's location on theGUI to Earth coordinates of the waypoint (414). As discussed in moredetail above with reference to FIG. 5, mapping (404) the pixel'slocation on the GUI to Earth coordinates of the waypoint (414) typicallyincludes mapping pixel boundaries of the GUI map to corresponding Earthcoordinates and identifying a range of latitude and a range of longituderepresented by each pixel. Mapping (404) the pixel's location on the GUIto Earth coordinates of the waypoint (414) also typically includeslocating a region on the surface of the Earth in dependence upon theboundaries, the ranges, and the location of the pixel on the GUI map.

The method of FIG. 12 also includes transmitting (406) uplink telemetry,including the coordinates of the waypoint, to the UAV through a socketon the remote control device. Transmitting (406) uplink telemetry,including the coordinates of the waypoint, to the UAV through a socketon the remote control device may be carried out by use of any datacommunications protocol, including, for example, transmitting thecoordinates as form data, URI encoded data, in an HTTP message, a WAPmessage, an HDML message, or any other data communications protocolmessage as will occur to those of skill in the art. Transmitting uplinktelemetry through a socket may be implemented by opening a socket,creating an output stream for the socket, and writing uplink telemetrydata to the output stream, as illustrated, for example, in the followingsegment of pseudocode:

uavSocket = new Socket( “computerAddress”, 7); outStream = newPrintWriter(uavSocket.getOutputStream( ), true);outStream.println(String someUplinkData);

This segment opens a socket object named “uavSocket” with an outputstream named “outStream.” Transmitting uplink telemetry through thesocket is accomplished with a call to outStream.println( ) which takesas a call parameter a String object named “someUplinkData.”

The method of FIG. 12 also includes receiving (408) downlink telemetry,including a starting position from a GPS receiver, from the UAV throughthe socket and piloting (410) the UAV, under control of a navigationcomputer on the UAV, from the starting position to the waypoint inaccordance with a navigation algorithm. Methods of piloting a UAVaccording to a navigation algorithm are discussed in detail below inthis specification.

Receiving downlink telemetry through a socket may be implemented byopening a socket, creating an input stream for the socket, and readingdata from the input stream, as illustrated, for example, in thefollowing segment of pseudocode:

uavSocket = new Socket( “computerAddress”, 7); inStream = newBufferedReader(new InputStreamReader(uavSocket.getInputStream( )));String downLinkTelemetry = inStream.readLine( );

This segment opens a socket object named “uavSocket” with an inputstream named “inStream.” Receiving downlink telemetry through the socketis accomplished with a blocking call to inStream.readLine( ) whichreturns a String object name “downLinkTelemetry.”

In the method of FIG. 12, downlink telemetry may include Earthcoordinates of waypoints as well as one or more UAV instructions. It iscounterintuitive that downlink telemetry contains waypoint coordinatesand UAV instructions when the expected data communications direction forwaypoint coordinates and UAV instructions ordinarily is in uplink from aremote control device to a UAV. It is useful to note, however, thatwaypoint coordinates and UAV instructions can be uplinked from amultiplicity of remote control devices, not just one. A flight linetechnician with a handheld PDA can issue waypoint coordinates and UAVinstructions to a UAV that is also linked for flight control to acomputer in a ground station. It is sometimes advantageous, therefore,for downlink telemetry to include waypoint coordinates or UAVinstructions so that one remote control device can be advised of thefact that some other remote control device issued waypoint coordinatesor UAV instructions to the same UAV.

Macros

As mentioned above, embodiments of the present invention often supportnavigating a UAV along a route having many waypoints, including a finalwaypoint and one or more intermediate waypoints. That is, methods of thekind illustrated in FIG. 12 may also include receiving user selectionsof a multiplicity of GUI map pixels representing waypoints, where eachpixel has a location on the GUI and mapping each pixel location to Earthcoordinates of a waypoint.

Such methods of navigating a UAV can also include assigning one or moreUAV instructions to each waypoint and transmitting the coordinates ofthe waypoints and the UAV instructions in the uplink telemetry throughthe socket to the UAV. A UAV instruction typically includes one or moreinstructions for a UAV to perform a task in connection with a waypoint.Exemplary tasks include turning on or off a camera installed on the UAV,turning on or off a light installed on the UAV, orbiting a waypoint, orany other task that will occur to those of skill in the art. Suchexemplary methods of navigating a UAV also include storing thecoordinates of the waypoints and the UAV instructions in computer memoryon the UAV, piloting the UAV to each waypoint in accordance with one ormore navigation algorithms (416), and operating the UAV at each waypointin accordance with the UAV instructions for each waypoint.

Navigation on a Course to a Waypoint

A further exemplary method of navigating in accordance with a navigationalgorithm is explained with reference to FIGS. 13, 14, and 15. FIG. 13sets forth a flow chart illustrating an exemplary method of piloting inaccordance with a navigation algorithm. FIG. 14 sets forth a linedrawing illustrating a method of calculating a heading with a cross windto achieve a particular ground course. And FIG. 15 sets forth a linedrawing illustrating a flight path produced by application of the methodof FIG. 13.

In the method of FIG. 13, piloting in accordance with a navigationalgorithm comprises periodically repeating (1212) the steps of reading(1202) from the GPS receiver a current position of the UAV; calculating(1204) a direction to the waypoint from the current position;calculating (1206) a heading in dependence upon wind speed, winddirection, airspeed, and the direction to the waypoint; turning (1208)the UAV to the heading; and flying (1210) the UAV on the heading.

FIG. 14 illustrates calculating a heading in dependence upon wind speed,wind direction, airspeed, and the direction to the waypoint. FIG. 14sets forth a line drawing illustrating relations among several pertinentvectors, a wind velocity (1222), a resultant velocity (1224), and aUAV's air velocity (1226). A velocity vector includes a speed and adirection. These vectors taken together represent wind speed, winddirection, airspeed, and the direction to the waypoint. In the exampleof FIG. 14, the angle B is a so-called wind correction angle, an anglewhich subtracted from (or added to, depending on wind direction) adirection to a waypoint yields a heading, a compass heading for a UAV tofly so that its resultant ground course is on a cross track. A UAVtraveling at an airspeed of ‘a’ on heading (D−B) in the presence of awind speed ‘b’ with wind direction E will have resultant ground speed‘c’ in direction D.

In FIG. 14, angle A represents the difference between the wind directionE and the direction to the waypoint D. In FIG. 14, the wind velocityvector (1222) is presented twice, once to show the wind direction asangle E and again to illustrate angle A as the difference between anglesE and D. Drawing wind velocity (1222) to form angle A with the resultantvelocity (1224) also helps explain how to calculate wind correctionangle B using the law of sines. Knowing two sides of a triangle and theangle opposite one of them, the angle opposite the other may becalculated, in this example, by B=sin⁻¹(b(sin A)/a). The two known sidesare airspeed ‘a’ and wind speed ‘b.’ The known angle is A, the angleopposite side ‘a,’ representing the difference between wind direction Eand direction to the waypoint D. Calculating a heading, angle F on FIG.14, is then carried out by subtracting the wind correction angle B fromthe direction to the waypoint D.

FIG. 15 shows the effect of the application of the method of FIG. 13. Inthe example of FIG. 15, a UAV is flying in a cross wind having crosswind vector (708). Curved flight path (1316) results from periodiccalculations according to the method of FIG. 13 of a new headingstraight whose resultant with a wind vector is a course straight from acurrent location to the waypoint. FIG. 15 shows periodic repetitions ofthe method of FIG. 13 at plot points (1310, 1312, 1314). For clarity ofexplanation, only three periodic repetitions are shown, although that isnot a limitation of the invention. In fact, any number of periodicrepetitions may be used as will occur to those of skill in the art.

Navigation on a Course Set to a Cross Track Direction

A further exemplary method of navigating in accordance with a navigationalgorithm is explained with reference to FIGS. 16 and 17. FIG. 16 setsforth a flow chart illustrating an exemplary method of piloting inaccordance with a navigation algorithm, and FIG. 17 sets forth a linedrawing illustrating a flight path produced by application of the methodof FIG. 16.

The method of FIG. 16 includes identifying (1402) a cross track andcalculating (1404) a cross track direction from the starting position tothe waypoint. In the method of FIG. 16, piloting in accordance with anavigation algorithm is carried out by periodically repeating the stepsof reading (1406) from the GPS receiver a current position of the UAV;calculating (1408) a shortest distance between the cross track and thecurrent position; and, if the shortest distance between the cross trackand the current position is greater than a threshold distance, piloting(1412) the UAV to the cross track. Upon arriving at the cross track, themethod includes: reading (1414) from the GPS receiver a new currentposition of the UAV; calculating (1416), in dependence upon wind speed,wind direction, airspeed, and the cross track direction, a new heading;turning (1418) the UAV to the new heading; and flying (1420) the UAV onthe new heading.

FIG. 17 shows the effect of the application of the method of FIG. 16. Inthe example of FIG. 17, a UAV is flying in a cross wind having crosswind vector (708). Curved flight path (1504) results from periodiccalculations according to the method of FIG. 16 of a shortest distancebetween a current position and the cross track (706), flying the UAVback to the cross track, and, upon arriving at the cross track,calculating a new heading (1502, 1505, and 1506) and flying the UAV onthe new heading.

Navigating a UAV Having an On-Board Digital Camera to Capture DesiredGeographic Area

As described above, UAVs according to the present invention ofteninclude an on-board digital camera useful in monitoring a geographicarea by the UAV. Such UAVs are capable of transmitting images capturedby the on-board digital camera to a remote control device or otherdisplay device, or are capable of storing the images for later review.The dimensions of the geographic area captured by an on-board digitalcamera will vary according to the current altitude of the UAV, theflying pattern of the UAV, camera settings of the UAV, as well as otherfactors that will occur to those of skill in the art.

FIG. 18 sets forth a flow chart illustrating an exemplary method fornavigating a UAV (100) having an on-board digital camera (552) thatincludes identifying (554) a geographic area (560) not captured by thedigital camera while the UAV (100) is flying in a current flying pattern(562) and modifying (558) the current flying pattern (562) to capture animage of the identified geographic area (560). As mentioned above, onefactor affecting the dimensions of a geographic area (560) not capturedby the on-board camera while the UAV is flying a current flying pattern(562) is the flying pattern itself. That is, the shape of the flyingpattern and the altitude of the UAV. A flying pattern is a consistentpattern of flight of a UAV. Flying patterns include patterns fororbiting a waypoint, flying a square identified by four cornercoordinates, or other shapes around waypoints that will occur to thoseof skill in the art.

In the example of FIG. 18, the UAV (100) is flying a square shapedpattern (562). The digital camera (552) in the example of FIG. 18 iscurrently capturing a geographic area (556) having a portion within thearea bounded by the current flying pattern and a portion outside thearea bounded by the current flying pattern (562). In the example of FIG.18, the current altitude and camera settings on the digital cameraleaves an uncaptured geographic area (560) when flying the currentsquare shaped pattern (562). The size and shape of an uncapturedgeographic area will typically depend upon the size and shape of thecurrent flying pattern of the UAV, as well as the size of the geographicarea captured by the camera.

Capturing digital images of a geographic area with a digital cameramounted on a UAV is facilitated by flying patterns. For furtherexplanation therefore, FIG. 19 sets forth a flow chart illustrating anexemplary method for flying a pattern. A flying pattern is implementedby a consistent series of flight control instructions identified usingalgorithms unique to the pattern that pilot the UAV such that theresulting flight path creates a pattern of a particular shape over theground at a particular altitude.

The method of FIG. 19 includes repeatedly receiving (650) from a GPSreceiver a current position of the UAV, calculating (654) a heading independence upon a current flying pattern algorithm (652), and flying(656) on the heading. Calculating (654) a heading in dependence upon acurrent flying pattern algorithm (652) may be carried out by anavigational computer on-board the UAV or by a navigational computer ina remote control device. The particular heading calculated for flying aparticular pattern will vary according to the flying pattern algorithmitself. For example, on algorithm for flying an orbit around a waypointincludes calculating a locus of points in a circle according to adefined radius and establishing a turn on that circle. One way tomaintain the orbit in the presence of cross wind includes establishing athreshold distance from the calculated circle and periodically adjustingthe heading of the UAV when the UAV deviates more that the thresholddistance from the calculated circle. A square shaped flying pattern maybe accomplished by defining four coordinates representing corners of thesquare and piloting the UAV to each of the four coordinates sequentiallyto fly a square.

The inclusion of a circular flying pattern and a square shaped flyingpattern are for explanation and not for limitation. In fact, UAVsaccording to embodiments of the present invention may fly patterns ofmany shapes as will occur to those of skill in the art includingcircles, squares defined by particular coordinates, and other polygonsas will occur to those of skill in the art.

As described above with reference to FIG. 18, on-board cameras provide alimited area coverage depending upon the altitude of the UAV, the cameraangle, the camera settings, and other factors that will occur to thoseof skill in the art. Identifying a geographic area not captured by thedigital camera while flying the UAV in a current pattern is useful inmodification of the altitude or flying pattern of the UAV to capture thepreviously uncaptured geographic area. For further explanation, FIGS. 20and 21 set forth line drawings illustrating an aerial view of geographicareas uncaptured by a UAV flying a square shaped pattern. The example ofFIG. 20 shows an aerial view of a square shaped flying pattern (562).The area (556) currently captured by the onboard camera is a geographicarea beneath the UAV the size of which will vary according to thealtitude of the UAV, the camera angle of the on-board digital camera,the camera settings of the camera, and other factors as will occur tothose of skill in the art. In the example of FIG. 20, the area captured(556) by the onboard camera extends within the area bounded by thecurrent flying pattern (563). The area captured (556) by the onboardcamera also extends beyond the perimeter of the current flying pattern(562).

To identify a geographic area not captured by the digital camera themethod of FIGS. 20 and 21 includes extrapolating the area captured bythe onboard camera along the flying pattern to determine a perimeter ofuncaptured geographic area. Turning now to FIG. 21, the total area (567)captured by an on-board camera beneath a UAV flying the square shapedflying pattern is obtained by extrapolating the area captured by thecamera along the flight path. Because portions of the area captured bythe camera may extend beyond the area bounded by the flight path, theouter perimeter (569) of the total area (567) captured by the on-boardcamera is often larger than the perimeter of the flight path itself. Theinner perimeter (571) of the total area (567) captured by the onboardcamera defines the outer boundary of the uncaptured geographic area(560).

Identifying a geographic area not captured by the digital cameraaccording to the methods of FIGS. 20 and 21 therefore includesdetermining the area of the uncaptured geographic area in dependenceupon the inner perimeter. Typically determining the area of theuncaptured geographic area in dependence upon the perimeter, is carriedout by determining the area bounded by the inner perimeter.

Identifying uncaptured geographic areas is not limited to situationswhere a single UAV is flying a pattern. In fact, often UAVs fly information with other UAVs leaving gaps between their respective areas ofcamera coverage. For further explanation therefore, FIG. 22 sets forth aline drawing illustrating two UAVs (110, and 564) each equipped withon-board digital cameras (552 and 566) and each flying a square shapedflying pattern (562 and 572). Each UAV has a currently capturedgeographic area (556 and 568) representing the geographic area currentlycaptured by the respective on-board digital camera. Identifying ageographic area not captured by the digital camera while the UAV (100)is flying in a current flying pattern (562) according to the example ofFIG. 22 therefore includes determining a gap (570) in camera coveragebetween the camera coverage (556) of the UAV (100) in the current flyingpattern (562) and the camera coverage (568) of another UAV (564) inanother flying pattern (572). In the example of FIG. 22, each digitalcamera (552 and 566) captures an area (556 and 568) that extends beyondthe respective perimeter of the flying pattern (562 and 572) of itscorresponding UAV.

For further explanation, FIG. 23 sets forth a line drawing illustratingan aerial view of geographic areas uncaptured by a two UAVs flyingsquare shaped patterns useful in explaining a method for identifying ageographic area not captured by the digital camera while the UAV isflying in a current flying pattern in formation with another UAV. In theexample of FIG. 23, the camera coverage of two UAVs is extrapolatedalong their flight respective flight paths resulting in the total area(567) captured by a first on-board camera on the first UAV and the totalarea (569) captured by the second on-board camera on the second UAV.Determining a gap (570) in camera coverage between the total cameracoverage (567) of the first UAV and the total camera coverage (569) ofthe second UAV according to the example of FIG. 23 includes determiningan area (570) between a portion (575) of the outer perimeter of thetotal area (567) captured by the first on-board camera and a portion(573) of the perimeter of the total area (569) captured by the secondon-board camera.

The example of two UAVs flying in square shaped patterns is forexplanation and not for limitation. In fact, UAVs according to thepresent invention may be implemented using a single UAV or many UAVsflying any pattern as will occur to those of skill in the art.

After identifying a geographic area not captured by the digital camera,methods of navigating a UAV according to the present invention includemodifying the current flying pattern to capture an image of theidentified and uncaptured geographic area. Modifying the current flyingpattern to capture an image of the identified geographic area may becarried out by identifying flight control instructions for changing theshape of the current flying pattern of the UAV, identifying flightcontrol instructions for increasing the altitude of the UAV, identifyingflight control instructions for changing the shape of the currentpattern, identifying flight control instructions for changing the scopeof the current flying pattern of the UAV by for example changing theradius of the pattern, or other ways as will occur to those of skill inthe art. Modifying the current flying pattern to capture an image of theidentified and uncaptured geographic area may be carried out by reducinga distance between the first UAV and the second UAV by for exampleidentifying flight control instructions for modifying one or more theflying patterns of the UAVs to reduce the distance between the UAVs.Flight control instructions may be identified and executed by anavigational algorithm on a navigational computer on-board the UAV.Alternatively, flight control instructions may be identified by anavigational algorithm on a navigational computer in a remote controldevice and transmitted to the UAV to modify the current flying pattern.

For further explanation, FIG. 24 sets forth line drawing illustrating achanged area of camera coverage resulting from modifying the flyingpattern of the UAV. In the example of FIG. 24, modifying the currentflying pattern to capture an image of the identified and previouslyuncaptured geographic area (560) includes identifying flight controlinstructions for changing the altitude (576) of the UAV (100). In theexample of FIG. 24, increasing the altitude (576) of the UAV (100)results in increasing size of the currently captured geographic area(574). The increased size of the currently captured geographic area(574) includes a portion of the previously uncaptured geographic area(560). In the example of FIG. 24, the shape of the previous flyingpattern (562) is not modified. Only the altitude of the UAV is modifiedsuch that at least a portion of the previously uncaptured geographicarea is now captured by the onboard digital (552) camera.

As discussed above, the shape of the flying pattern of a UAV may also bechanged to capture the previously uncaptured area. For furtherexplanation, FIG. 25 sets forth a line drawing illustrating the changedarea of camera coverage resulting in changing the flying pattern of theUAV by changing the shape of the flying pattern of the UAV. In theexample of FIG. 25, the modified flying pattern of the UAV (100) ischanged from a rectangular flying pattern (562) to a circular flyingpattern that orbits a waypoint (561) located generally in the center ofthe previously uncaptured geographic area (560). Modifying the flyingpattern advantageously results in an area (574) captured by the on-boardcamera that covers the previously uncaptured area (560).

It will be understood from the foregoing description that modificationsand changes may be made in various embodiments of the present inventionwithout departing from its true spirit. The descriptions in thisspecification are for purposes of illustration only and are not to beconstrued in a limiting sense. The scope of the present invention islimited only by the language of the following claims.

1. A system for navigating a UAV having an on-board digital camera, thesystem comprising: means for piloting the UAV in a current flyingpattern; wherein means for piloting the UAV in a current flying patterncomprises; means for receiving from a GPS receiver a current position ofthe UAV; means for calculating a heading in dependence upon a flyingpattern algorithm; and means for flying on the heading; means foridentifying a geographic area not captured by the digital camera whilethe UAV is flying in a the current flying pattern; wherein means foridentifying a geographic area not captured by the digital camera whilethe UAV is flying in a current flying pattern further comprises: meansfor determining an area captured by the onboard camera; means forextrapolating the area captured by the onboard camera along the flyingpattern to determine a perimeter of uncaptured geographic area; andmeans for determining the area of the uncaptured geographic area independence upon the perimeter; and means for modifying the currentflying pattern to capture an image of the identified geographic area. 2.The system of claim 1 wherein means for identifying a geographic areanot captured by the digital camera while the UAV is flying in a currentflying pattern further comprises means for determining a gap in cameracoverage between the camera coverage of the on-board camera of the UAVin the current flying pattern and a camera coverage of another on-boardcamera of another UAV in another flying pattern.
 3. The system of claim1 wherein means for modifying the current flying pattern to capture animage of the identified geographic area further comprises means foridentifying flight control instructions for changing the altitude of theUAV.
 4. The system of claim 1 wherein means for modifying the currentflying pattern to capture an image of the identified geographic areafurther comprises means for identifying flight control instructions forchanging the shape of the current flying pattern of the UAV.
 5. Acomputer program product for navigating a UAV having an on-board digitalcamera, the computer program product comprising: a recording medium;means, recorded on the recording medium, for piloting the UAV in acurrent flying pattern; wherein means, recorded on the recording medium,for piloting the UAV in a current flying pattern comprises: means,recorded on the recording medium, for receiving from a GPS receiver acurrent position of the UAV; means, recorded on the recording medium,for calculating a heading in dependence upon a flying pattern algorithm;and means, recorded on the recording medium, for flying on the heading;means, recorded on the recording medium, for identifying a geographicarea not captured by the digital camera while the UAV is flying in thecurrent flying pattern; wherein means, recorded on the recording medium,for identifying a geographic area not captured by the digital camerawhile the UAV is flying in a current flying pattern further comprises:means, recorded on the recording medium, for determining an areacaptured by the onboard camera; means, recorded on the recording medium,for extrapolating the area captured by the onboard camera along theflying pattern to determine a perimeter of uncaptured geographic area;and means, recorded on the recording medium, for determining the area ofthe uncaptured geographic area in dependence upon the perimeter; andmeans, recorded on the recording medium, for modifying the currentflying pattern to capture an image of the identified geographic area. 6.The computer program product of claim 5 wherein means, recorded on therecording medium, for identifying a geographic area not captured by thedigital camera while the UAV is flying in a current flying patternfurther comprises means, recorded on the recording medium, fordetermining a gap in camera coverage between the camera coverage of theon-board camera of the UAV in the current flying pattern and a cameracoverage of another on-board camera of another UAV in another flyingpattern.
 7. The computer program product of claim 5 wherein means,recorded on the recording medium, for modifying the current flyingpattern to capture an image of the identified geographic area furthercomprises means, recorded on the recording medium, for identifyingflight control instructions for changing the altitude of the UAV.
 8. Thecomputer program product of claim 5 wherein means, recorded on therecording medium, for modifying the current flying pattern to capture animage of the identified geographic area further comprises means,recorded on the recording medium, for identifying flight controlinstructions for changing the shape of the current flying pattern of theUAV.